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ABSTRACT 



Since Engineering Criteria 2000 (EC 2000) was first 



introduced as the standard for American engineering education programs, most 
discussion in the literature has focused on how to assess Outcomes 3a-3k 
(specific outcomes or abilities engineering students must acquire as defined 
by the Accreditation Board of Engineering and Technology) and relatively 
little has concerned how to equip students with the skills and attitudes 
specified in those outcomes. This paper seeks to fill this gap. Following an 
overview of EC 2000 and an attempt to clarify the confusing array of terms 
that it has introduced into the accreditation process, the paper addresses 
two primary questions: (1) How can learning objectives, learning assessment 

methods, and instructional techniques for individual courses be formulated to 
address each of the Criterion 3 outcomes? and (2) What might be done at the 
program and individual course levels to raise the level of achievement of the 
outcomes? In the procedure recommended in the paper, once a program faculty 
has defined the program educational objectives, outcomes, and outcome 
indicators, it should: (1) identify the core (required) courses that address 

the outcomes; (2) define observable learning objectives for each core course 
that address the relevant program outcomes; and (3) select methods to assess 
the objectives. (Observable objectives involve action verbs like list, 
explain, calculate, model, design, and critique, as opposed to more abstract 
words like learn, understand, and appreciate. Some objectives can be assessed 
using written problem-solving tests; others cannot.) The courses should then 
be taught so as to maximize the likelihood that students capable of achieving 
the objectives will in fact do so. The paper provides resources to support 
this process, including illustrative learning objectives and instructional 
methods that address Outcomes 3a-3k. The methods suggested in the paper have 
all been used extensively and are well supported by educational research. 
(Evidence for this claim is provided by numerous references cited in the 
paper.) The intention is for course instructors to browse through the 
suggestions and select learning objectives and teaching and assessment 
methods that address their program outcomes and are compatible with their 
teaching philosophies and levels of experience. As time goes by, they should 
periodically review their assessment results, revisit the suggestions, and 
consider modifying their teaching and assessment methods in an effort to 
improve the level of achievement of the objectives. Appended are: 

Illustrative Learning Objective for EC 2000 Outcomes 3a-3k; Teaching Methods 
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that Address Outcomes 3a- 3k; Cooperative Learning Methods that address 
outcomes 3a-3k; and Problem-Based Learning Methods that Address Outcomes 
3a-3k. (Contains 91 references.) (Author/YDS) 
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ABSTRACT 



Since Engineering Criteria 2000 was first introduced to American engineering education, most discussion 
in the literature has focused on how to assess Outcomes 3a-3k and relatively little has concerned how to 
equip students with the skills and attitudes specified in those outcomes. This paper seeks to fill this gap. 
Following an overview of EC 2000 and an attempt to clarify the confusing array of terms that it has 
introduced into the accreditation process (objectives, outcomes, outcome indicators, etc ), the paper 
addresses the following questions: 

1 ffow can learning objectives , learning assessment methods , and instructional techniques for 
individual courses be formulated to address each of the Criterion 3 outcomes? 

2. What might be done at the program and individual course levels to raise the level of achievement 
of the outcomes ? 

In the procedure recommended in the paper, once a program faculty has defined the program 
educational objectives, outcomes, and outcome indicators, it should (a) identify the core (required) 
courses that address the outcomes, (b) define observable learning objectives for each core course that 
address the relevant program outcomes, and (c) select methods to assess the objectives. (Observable 
objectives involve action verbs like list, explain, calculate, model, design, and critique, as opposed to 
more abstract words like learn, understand, and appreciate. Some objectives can be assessed using 
written problem-solving tests; others cannot.) The courses should then be taught so as to maximize the 
likelihood that students capable of achieving the objectives will in fact do so. The paper provides 
resources to support this process, including illustrative learning objectives and instructional methods that 

address Outcomes 3a-3k. 

The methods suggested in the paper have all been used extensively and are well supported by 
educational research. (Evidence for this claim is provided by numerous references cited in the paper.) The 
intention is for course instructors to browse through the suggestions and select learning objectives and 
teaching and assessment methods that address their program outcomes and are compatible with their 
teaching philosophies and levels of experience. As time goes by, they should periodically review their 
assessment results, revisit the suggestions, and consider modifying their teaching and asssessment 
methods in an effort to improve the level of achievement of the objectives. 
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DESIGNING AND TEACHING COURSES 
TO SATISFY ENGINEERING CRITERIA 2000 



Richard M. Felder 
Rebecca Brent 

I. Introduction 

Engineering Criteria 2000 — the accreditation standard for all American engineering programs since the 
beginning of 2001 — has been the subject of extensive discussion since it was first introduced in 1996, and 
the intense nationwide curricular revamping that EC 2000 has catalyzed will inevitably lead to dramatic 
changes in engineering education. However, the potential of the new system to improve instruction is 
currently limited by several commonly expressed faculty misconceptions: 

• They 've just replaced one bean-counting system with another one. We can still get by if 
we get some course materials to the ABET coordinator a few months before the visit and 
let her and the Department Head work up the self-study. 

« We just need to copy Outcomes a-k right out of the book and say we 're addressing them 
and we ’ll be fine. 

• All we need to do to meet the new requirements is tack some methods for assessing 
communication skills, global awareness, and lifelong learning skills onto the things 
we ’ve always done. 

As some programs have learned to their sorrow, these lines of reasoning don’t sit well with ABET 
visitors. The reality is that beginning to plan for EC 2000 a few months before a visit, copying Outcomes 
3a-3k verbatim rather than refraining them to address program-specific objectives, and simply assessing 
program outcomes without putting in place a mechanism to improve the program based on the assessment 
results, are all prescriptions for failure to obtain full accreditation. 

Under the old system, the burden of preparing for an ABET visit did in fact reside almost 
exclusively with the accreditation coordinator, who did most of the work in putting together the self-study 
report and preparing a display for the visitor. Not any more! In the words of Jack Lohmann [1999], 
“Preparing for an ABET visit is no longer the academic equivalent of El Nino — something to be 
weathered every six years until things go back to normal.” Since the work of equipping students with the 
attributes specified in program outcomes must be done at the individual course level, all faculty members 
involved in teaching required courses must now understand and be involved in the accreditation process 
on a continuing basis, not just in the months preceding each visit. 

Understanding EC 2000 is no trivial goal, however: the jargon (objectives, outcomes, outcome 
indicators, performance targets, etc.) is dense and confusing, and universally agreed-upon operational 
definitions of the terms do not yet exist. Moreover, while much has been written in the past few years 
about the assessment of program outcomes (more specifically, of Outcomes 3a-3k), relatively little 
attention has been paid so far to the central role of the individual faculty member in attaining those 
outcomes. The purpose of this paper is to examine that role. 

Many of the programmatic requirements of EC 2000 are similar to those of the old system and are 
laid out reasonably well in the documentation on the ABET web site [ABET 2000], with most of the 



departures from prior practice occurring in Criteria 2 (program objectives) and 3 (program outcomes and 
continuous program improvement). Our focus in the paper will therefore be on those two criteria. In 
Section II, we overview EC 2000, attempt to clarify the terms that regularly appear in the EC 2000 
literature, and briefly review the mandated procedure for formulating program educational objectives. In 
Sections III— VI we assume that a program has formulated its objectives and compatible outcomes that 
encompass Outcomes a-k of Criterion 3, and address the following questions: 

1. How can learning objectives, assessment methods , and instructional techniques for individual 
courses be formulated to address each of the Criterion 3 outcomes? Such a formulation is a 
necessary condition for addressing the program outcomes. 

2. What might be done at the program and individual course levels to raise the level of achievement of 
the outcomes ? Doing so would address the requirement for continuous program improvement 
mandated by Criterion 3. 

The planning, teaching, and assessment methods we will present are neither novel nor radical, all of them 
have been used extensively and are well supported by educational research. The paper briefly surveys the 
methods and cites sources of more detailed information about them and the research that supports them. 



D. Elements of EC 2000 



A. Overview 

To comply with EC 2000, a program must first formulate program educational objectives (broad goals) 
that address institutional and program mission statements and are responsive to the expressed interests of 
various groups of program stakeholders. The program must then formulate a set of program outcomes 
(knowledge, skills, and attitudes the program graduates should have) that directly address the educational 
objectives and encompass certain specified outcomes (Outcomes 3a-3k, shown in Table 1). In required 
courses, core course learning objectives (statements of specific things students who pass the course 
should be able to do to demonstrate what they have learned) that address the program outcomes must be 
written, along with specifications of which outcomes they address. The program objectives and outcomes 
must be set forth in a self-study report, which must also include statements of where the outcomes are to 
be addressed in the program curriculum, how their level of attainment is to be assessed, and how the 
assessment results will be used to improve the program. Beginning with the second accreditation visit 
under EC 2000, the program will also presumably have to demonstrate that it has implemented the 
improvement plan formulated in the prior visit. 

When first confronted with EC2000, faculty members have an understandable inclination to 
formulate their program objectives and outcomes to fit their existing curricula. This approach is 
invariably frustrating and possibly self-defeating. Existing curricula, lacking any historical reason to have 
been scrutinized in the light of desired learning outcomes, are often little more than collections of content- 
driven courses that have only the loosest of connections to one another. This disjointedness is reflected in 
the blank stares of incomprehension familiar to all engineering faculty members who have ever asked 
their students about material from a prerequisite or corequisite course. Tailoring the accreditation process 
to perpetuate the status quo will clearly not improve this situation. 

EC 2000 is an antidote to curricular chaos. The exercise of constructing a clear program mission, 
broad goals that address the mission (program educational objectives), and desired attributes of the 
program graduates (program outcomes) requires the faculty to consider seriously — possibly for the first 
time — what their program is and what they would like it to be. The product of this exercise constitutes a 
unifying framework for course and curriculum development. If faculty members then structure their 
course syllabi, learning objectives, and teaching and assessment methods to address the program 



outcomes, the result is a coherent curriculum in which all courses have well-defined and interconnected 
roles in achieving the program mission. The learning objectives — explicit statements of what students in 
a course are expected to do to demonstrate their mastery of course material are crucial to the process, 
among other things, they enable the program to demonstrate precisely how specific program outcomes are 
addressed in the curriculum. If the outcomes are then assessed continuously and the results are used to 
improve instruction in the courses that address them, the degree to which the program meets its self- 
selected goals must inevitably improve. 

When a program approaches accreditation in this logical manner, the burden of preparing the self- 
study may actually be decreased from what it was under the old system. Lohmann [1999] reports that the 
self-study for the B.S. program in Mechanical Engineering at Georgia Tech occupied 576 pages in 1990 
under the old system and only 180 pages in 1997 under EC 2000, of which 130 pages comprised the 
required faculty resumes and course outlines. 

Creating a course to achieve specified outcomes requires effort in three domains (see Figure 1): 
design (identifying course content and defining measurable objectives), instruction (selecting and 
implementing the methods that will be used to deliver the specified content and facilitate student 
achievement of the objectives), and assessment and evaluation (selecting and implementing the methods 
that will be used to determine whether and how well the objectives have been achieved and interpreting 
the results). As Figure 1 shows, the three stages are not purely sequential— the information collected in 
each of them feeds back to each of the others in a cycle that leads to continuous improvement. If the 
assessment reveals that an objective has not been satisfactorily achieved, the nature of the failure may 
suggest augmenting coverage of certain topics in the syllabus or formulating more explicit objectives 
related to those topics. Similarly, as the quality of the instructional program improves, new or modified 
objectives may be formulated to encompass higher levels of achievement and course instruction and 
outcomes assessment modified to fit the objectives. 

B. Terminology 

A particularly confusing aspect of EC 2000 is its use of normally interchangeable terms such as goals, 
outcomes, and objectives to mean different things. The definitions that follow will be used in this paper. 

1 Program educational objectives (or program goals) — “broad, general statements that communicate 
how an engineering program intends to fulfill its educational mission and meet its constituencies 
needs.” [Aldridge & Benefield 1998] 

ExamDle- Provide students with a solid grounding in the basic sciences and mathematics, an 
understanding and appreciation of the arts, humanities, and social sciences, and proficiency m 
both engineering science and design. 

2. Program outcomes— more specific statements of program graduates’ knowledge, skills, and 
attitudes that serve as evidence of achievement of the program s educational objectives. 

Example: The program graduates will be able to analyze important social and environmental 
problems and identify and discuss ways that engineers might contribute to solutions, including 
technological, economic, and ethical considerations in their analysis. 

In Criterion 3 of EC 2000, ABET specifies 1 1 required outcomes (Outcomes 3a-3k, listed in Table 
1) Program outcomes must encompass Outcomes 3a-3k but should not be verbatim copies of them. 
To meet the requirements of EC 2000, the program outcomes should clearly have been crafted by the 
faculty to address all of the program educational obj ectives . 
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3. Outcome indicators — the measuring instruments and methods that will be used to assess the 
students’ attainment of the program outcomes [Scales et al. 1998], 

Examples : Alumni, employer, and industrial advisory board surveys, exit interviews with 
graduating seniors, student portfolios, capstone design course performance ratings, performance 
on standardized tests like the FE Examination and the GRE, and job placement data of graduates. 

4. Performance targets — the target conditions for the outcome indicators. 

Examples : 

o The [average score, score earned by at least 80%] of the program graduates on the 
[standardized test, standardized test item, capstone design report, portfolio evaluation ] must 
be at least 75/100. 

O The [median rating for, rating earned by at least 80% of] the program graduates on the [self- 
rating sheet, peer rating sheet, senior survey, alumni survey, employer survey, final oral 
presentation ] must be at least [75/100, 4.0 on a 1-5 Likert scale, Very good ]. 

5. Outcome elements— different abilities specified in a single outcome that would generally require 
different assessment measures. As part of an ongoing study, Besterfield-Sacre et al. [2000(a)] break 
each of Outcomes 3a-3k into separate elements. For some outcomes, such as Outcome 3b, the 
elements are literally extracted from the outcome statement: 

Outcome 3b — ability to design and conduct experiments, as well as analyze and interpret data => 
designing experiments, conducting experiments, analyzing data, interpreting data. 

For others, such as Outcome 3e, the elements are derived from an analysis of the specified abilities. 

Outcome 3e — ability to identify, formulate, and solve engineering problems => problem 
identification, problem statement construction and system definition, problem formulation and 
abstraction, information and data collection, model translation, validation, experimental design, 
solution development or experimentation, interpretation of results, implementation, 
documentation, feedback and improvement. 

6. Outcome attributes— actions that explicitly demonstrate mastery of the abilities specified in an 
outcome or outcome element. The main thrust of the work of Besterfield-Sacre et al. [2000(a)] is to 
define attributes at the six levels of Bloom’s taxonomy of cognitive objectives [Bloom et al. 1984] 
and at the valuation level of Krathwohl’s taxonomy of affective objectives [Krathwohl et al. 1984] for 
each of Outcomes 3a-3k.. 

Examples : Attributes proposed by Besterfield-Sacre et al. [2000(a)] for the element Problem 
statement construction and system definition” of Outcome 3e (see above) include 

o describes the engineering problem to be solved 

« visualizes the problem through sketch or diagram 

• outlines problem variables, constraints, resources, and information given to construct a 
problem statement 

• appraises the problem statement for objectivity, completeness, relevance, and validity 
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7. Course outcomes— knowledge, skills, and attitudes that the students who complete a course should 
acquire. Some of the outcomes in each required course should map onto (or be identical with) one or 
more program outcomes. 

8. Course learning objectives (or instructional objectives) — statements of observable student actions 
that serve as evidence of the knowledge, skills, and attitudes acquired in a course. 

Examples : The students will be able to 

o explain in terms a high school student could understand the concepts of specific gravity, 
vapor pressure, and dew point 

O solve a second-order ordinary differential equation with specified initial conditions using 
Matlab 

o design and carry out an experiment to measure a tensile strength and determine a 95/o 
confidence interval for its true value 

• define the four stages of team functioning and outline the responsibilities of a team 
coordinator, recorder, checker, and process monitor 

Learning objectives should begin with observable action words (such as explain, outline, 
calculate, model, design, and evaluate) and should be as specific as possible, so that an observer would 
have no trouble determining whether and how well students have accomplished the specified task. Words 
like “know,” “learn,” “understand,” and “appreciate” may be suitable for use in educational objectives or 
program outcomes but not learning objectives. To know whether or not students understand, say, the 
impact of engineering solutions in a global/societal context (Outcome 3h), one must ask them to do 
something to demonstrate that understanding, such as identify an important problem and discuss ways 
engineers might help solve it. 

We have defined these terms because they all appear in the accreditation literature and in 
published self-study reports, but there is no requirement that any individual self-study make use of all of 
them. The only ones mentioned by ABET are the first two (program educational objectives and program 
outcomes) and the last one (course learning objectives); the other terms might or might not be included in 
a program self-study, depending on how the program chooses to approach EC 2000. 



C. Criterion 2 and Program Educational Objectives 



The ABET accreditation criteria in place before 200 1 spelled out in fairly precise terms what was required 
for a program to be accredited (so many credits of engineering science, engineering design, humanities, 
etc., and adequate faculty and resources to meet the educational needs of the student body). The rigidly 
prescriptive nature of the system was a source of frustration to engineering faculty and administrators, and 
the derisive label “bean counting” almost invariably arose whenever ABET came up in conversation. 

Requirements related to faculty and resources are still in place in EC 2000 Criteria 1 (advising 
students and monitoring their progress toward meeting program requirements), 4 (requirements regarding 
mathematics, science, design, various aspects of professional practice, and general education), 5 (faculty 
qualifications), 6 (facilities), 7 (institutional support and financial resources), and 8 (additional criteria for 
specific programs, normally formulated by the appropriate professional societies) [ABET 2000], On the 
other hand, Criteria 2 (program educational objectives) and 3 (program outcomes and assessment) are 
dramatic departures from prior practice. These criteria are largely non-prescriptive, a feature 
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characterized by some as “flexible” and by others as “fuzzy.” Recognizing that different engineering 
programs have different missions, student demographics, and resources, ABET leaves it mainly up to 
individual programs to define their own educational objectives, outcomes, learning objectives, 
instructional methods, and assessment procedures. 

Criterion 2 of EC 2000 states that a program seeking accreditation must (a) publish and 
periodically evaluate a set of educational objectives consistent with the institutional mission and the needs 
of the program constituencies, (b) implement a curriculum and process to achieve the objectives, and (c) 
put in place a system of ongoing evaluation to demonstrate the achievement of the objectives and 
continuously improve the program effectiveness [ABET 2000; Aldridge & Benefield 1998; Sarin 1998; 
Carter, Brent, & Rajala 2001; McGourty, Sebastian & Swart 1998], 

Carter, Brent, and Rajala [2001] offer guidance on how to meet Criterion 2. They suggest that 
programs seeking accreditation assemble university, college, and program/department mission statements, 
define the key stakeholders in the program (e.g., students, faculty, alumni, employers of program 
graduates, and funding sources), solicit their views on desirable program attributes, and write educational 
objectives that take into account the various mission statements and stakeholder desires. The content of 
the educational objectives is not subject to challenge by ABET, as long as the formulation guidelines 
prescribed by Criterion 2 were clearly observed. It follows that objectives might differ considerably from 
one program to another at a single institution (e.g., the construction option and the transportation option in 
a civil engineering department) and within a single program discipline (e.g., mechanical engineering) 
across universities. Additional suggestions for formulating educational objectives and documenting steps 
taken to address Criterion 2 are offered by Carter et al. [2000] and McGourty et al. [1998]. 

D. Criterion 3 and Outcomes 3a-3k 

Once program educational objectives are in place, program outcomes must be defined that specify the 
knowledge, skills, and attitudes program graduates should have if the educational objectives are achieved. 
As noted previously, the program outcomes must encompass the 1 1 outcomes specified in EC 2000 
Criterion 3 (see Table 1) but would normally go beyond them to address the complete set of program 
educational objectives. 

Besides listing Outcomes 3a— 3k, Criterion 3 requires programs seeking accreditation to have (1) 
an assessment process for the program outcomes, (2) documented results from the implementation of the 
process, and (3) “evidence that the results are applied to the further development and improvement of the 
program [ABET 2000].” An important corollary of this statement is that programs do not have to meet all 
of their outcome performance targets to be accredited (at least at the first accreditation visit under EC 
2000). They must only demonstrate that they have in place a sound plan for outcomes assessment and 
continuous program improvement and are making a serious effort to implement it. When ABET returns 
to re-evaluate the accredited program, the program will presumably have to show that it has made 
substantial progress with the implementation. 

At this point we are ready to proceed with the main topic of this paper: how engineering courses 
might be designed, taught, and assessed to equip students with the skills specified in EC 2000 Outcomes 
3a-3k. Outcomes assessment has been discussed at great length in the literature and so we will spend 
relatively little time on it, concentrating most of our attention on the less thoroughly examined topics of 
course design (specifically, formulation of learning objectives) and instruction. 
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TIT . Designing Courses to Address EC 2000 Criterion 3 

A. A Matrix-Based Structure for Course and Program Assessment 

Suppose that educational objectives have been formulated for an engineering program following the 
specifications of Criterion 2, and program outcomes that encompass Outcomes 3a-3k have in turn been 
formulated to address the educational objectives. The next step is to identify the program core — the 
courses in the program curriculum collectively designated to address the knowledge, skills, and attitudes 
specified in the program outcomes. Required courses under the control of the program (e.g., chemical 
engineering courses required of all chemical engineering majors) are obvious candidates for the core. 
Required courses given by other programs (e.g., mathematics and humanities courses) may also be 
included, provided that their published content consistently addresses program outcomes. Elective 
courses and courses whose content changes significantly from one instructor to another should not be 
included in the core. 

For every core course, a set of core course learning objectives should be defined that map onto 
one or more of the program outcomes and are guaranteed to be in place in all offerings of the course. 
Additional learning objectives might be (and normally would be) defined by individual instructors to 
reflect specific program requirements and their own personal goals, but the core objectives should be 
invariant. The program can then reasonably claim that if one or more core objectives in a course address 
a particular program outcome, the course addresses the outcome, which is precisely the sort of 
information that ABET evaluators look for in the self-study. If the course is taught outside the program, 
having the presenting department sign off on the core learning objectives can strengthen the claim. 

To keep track of how and where program outcomes are addressed in the curriculum, a course 
assessment matrix might be constructed for each core course, with a column for each program outcome 
and a row for each core learning objective. Entries of 1, 2, and 3 inserted in a cell of the matrix 
respectively indicate that an objective addresses an outcome slightly, moderately, or substantively. Table 
2 shows such a matrix. Once the course assessment matrices have been prepared, a program outcome 
assessment matrix can be constructed as shown in Table 3, with columns for program outcomes and rows 
for program outcome indicators and core courses. Entries of 1, 2, and 3 in the matrix respectively denote 
slight, moderate, and substantive relevance of the outcome indicators and core courses to the program 
outcomes. This matrix provides a concise summary of how the program outcomes are assessed and the 
courses to concentrate on when attempting to raise the attainment level of a particular outcome. The 
entries for a course should be based on a review of course materials (syllabi, learning objectives, tests and 
other assessment measures, and the course assessment matrix) conducted by a committee that includes all 
faculty members who teach the course. 

A common concern of ABET evaluators has to do with outcomes addressed in only one or two 
courses (especially if the courses are taught outside the program), such as communication skills addressed 
only in one or two general education courses and safety or ethics addressed only in the capstone design 
course. Programs are advised to distribute their coverage of each outcome throughout the program, not 
only for appearance’s sake but to provide repeated practice and feedback in the skills the students will 
need to meet the outcome performance target. 

Once the program assessment has been carried out, asterisks may be placed next to matrix entries 
for outcome indicators in a copy of the program assessment matrix to indicate that the relevant 
performance targets have been met. (This information would not necessarily be included in the self-study 
but could serve for internal program use only.) Entries without asterisks would identify possible focal 
points for the continuous improvement effort mandated by EC 2000 Criterion 3. 




7 



li 



B. Formulating Core Course Objectives 

Consider the illustrative program outcome given in Section II: 

The program graduates will be able to analyze important social and environmental 
problems and identify and discuss ways that engineers might contribute to solutions, 
including technological, economic, and ethical considerations in their analysis. 

It might reasonably be claimed that EC 2000 Outcomes 3e (identifying engineering problems), 3f 
(understanding professional and ethical responsibility), 3h (understanding global and societal implications 
of engineering solutions), and 3j (knowledge of contemporary issues) all map onto this hypothetical 
program outcome. To meet the requirements of EC 2000, each of Outcomes 3a-3k must map onto one or 
more program outcomes in this manner, from which it follows that the core courses in the program 
curriculum must collectively include core learning objectives that address each of Outcomes 3a-3k. 
(There is no need for any individual course to address all 1 1 outcomes, however, and it would be a rare 
course indeed that could do so.) 

Illustrative learning objectives for each of Outcomes 3a-3k are given in Appendix 1, and an 
extensive list of attributes for each element of each outcome (see Section II-A) is provided by Besterfield- 
Sacre et al. [2000(a)], Instructors seeking to formulate objectives for their courses may begin by adapting 
items from either of these sources. If the attributes in the second list are not directly expressed in 
measurable terms (e.g., if they begin with words like “know” or “understand” or “appreciate,”), the 
instructors may recast them using appropriate action words, many of which are also listed by Besterfield- 
Sacre et al. 



IV. Assessing Learning 

As soon as program outcomes, outcome indicators, and performance targets have been defined and core 
learning objectives have been drafted for all core courses, plans should be made for assessing the degree 
to which the learning objectives are being met. Expressing the objectives in terms of observable action 
words greatly simplifies this task. The assessment plan should also specify who is responsible for each 
part of the assessment, when the assessment will be performed, and who will receive the results [Sarin 
1998], 



Triangulation (using multiple methods to obtain and verify a result) is an important feature of 
effective assessment [Besterfield-Sacre et al. 2000(b)], The more tools used to assess a specific program 
outcome or course learning objective, the greater the likelihood that the assessment will be both valid 
(what is actually being assessed is what is supposedly being assessed) and reliable (the conclusion would 
be the same if the assessment were conducted by other assessors or again by the same assessor). 
Following are some possible program-level (P) and course-level (C) assessment tools: 

1. Exit surveys, exit interviews (P) 

2. Alumni surveys and interviews (P) 

3. Employer surveys and interviews (P) 

4. Job offers, starting salaries (relative to national benchmarks) (P) 

5. Admissions to graduate school (P) 

6. Performance in co-op and internship assignments and in problem-based learning situations (P,C) 

7. Assignments, reports, and tests in the capstone design course (P,C) 
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8. Standardized tests — e.g., the FE Examination (the value of which is discussed by Watson 
[1998]), the GRE, and the Force Concept Inventory in physics (P,C) 

9. Student surveys , individual and focus group interviews (P,C) 

10. Self-analyses, learning logs, journals (P,C) 

11. Peer evaluations, self-evaluations (P,C) 

12. Student portfolios (P,C) 

13. Behavioral observation, ethnographic and verbal protocol analysis (analyzing transcripts of 
student interviews or working sessions to extract patterns of problem-solving, thinking, or 
communication) (P,C) 

14. Written tests or test items clearly linked to learning objectives (C) 

15. Written project reports (C) 

16. Oral presentations (live or on videotape) (C) 

17. Research proposals, student-formulated problems (C) 

18. Abstracts, executive summaries, papers (C) 

19. Letters, memos (C) 

20. Written critiques of documents or oral presentations (C) 

21. Classroom assessment techniques [Angelo & Cross 1993; National Institute for Science 
Education (b), Shaeiwitz 1998] (C) 

Prus and Johnson [1994] and Nichols [1995] summarize the strengths and weakness of many of these 
assessment tools. 

Objective tests such as standardized multiple-choice and (setting aside questions related to partial 
credit) quantitative problem-solving tests may be evaluated reliably with relatively little difficulty, 
however, evaluation of a result obtained using any other assessment tool is at least in part a matter of the 
evaluator’s opinion, and achieving reliable ratings can be a problem. An effective approach is to identify 
aspects of the product or presentation to be rated (e.g., for grading project or laboratory reports, the 
aspects might be technical soundness, organization, thoroughness of discussion, and quality of writing), 
select a weighting factor for each aspect, and construct a rubric — a form on which the evaluator assigns 
numerical ratings to each specified aspect and then uses them and the weighting factors to compute an 
overall rating. Trevisan et al. [1999] offer suggestions regarding the effective design and use of rating 
forms including a recommendation that the characteristics of the highest and lowest ratings and the 
midpoint rating for each feature be spelled out fairly explicitly. If several raters complete forms 
independently and then reconcile their ratings, the result should be very reliable, and the reliability can be 
increased even further by giving raters preliminary training on sample products or videotaped 
presentations. 

Considerable expertise is required to design valid questionnaires and interviews [Dobson 1996; 
Fowler 1993; Sudman & Bradbum 1996], so unless an already validated instrument is available, 
assistance from a knowledgeable consultant should be sought when using these assessment tools. 
Similarly, assembling and evaluating student portfolios is a complex and time-consuming task that can 
become completely unmanageable without careful planning. Several resources are available to assist in 
this planning for portfolios in general [Barton & Collins 1997; Christy & Lima 1998; Olds & Miller 1997; 
Panitz 1996] and for electronic portfolios [Barrett 1998; Rogers & Williams 1999; Wiedmer 1998], 




9 



More detailed discussions of assessment in the context of EC 2000 are given by Besterfield-Sacre 
et al. [2000(a), 2000(b)], McGourty et al. [1999], Olds and Miller [1998], Rogers [2000], Rogers and 
Sando [1996], Scales et al. [1998], and Besterfield-Sacre et al. [2000]. Deek et al. [1999] discuss the 
assessment of problem-solving skills. 

V. Teaching to Address Outcomes 3a-3k 

EC 2000 has been discussed extensively in the engineering education literature since it first appeared, but 
most of the discussion has focused on assessing Outcomes 3a-3k, with relatively little being said about 
what must be done to achieve those outcomes. The tacit assumption seems to be that determining 
whether or not students have specific skills is much harder than equipping them with those skills. In fact, 
the opposite is closer to the truth. We know a great deal about how to assess communication skills, for 
example, but judging from the common complaints that most engineering graduates cannot write a 
coherent report or give a comprehensible talk, we clearly have not yet worked out how to raise those 
skills to satisfactory levels. 

In Subsection A we outline instructional methods that address each of the Criterion 3 outcomes 
and cite references on how to implement them and on the research that supports their effectiveness. 
Subsections B and C discuss the instructional paradigms of cooperative learning and problem-based 
learning, each of which has the potential to address all eleven Criterion 3 outcomes effectively. For 
additional descriptive details and research findings about the methods to be described, see Bransford et al. 
[2000], McKeachie [1999] and Wankat [2002]. 

A. General Instructional Methods 

The more explicitly students know what they are expected to do and the more practice they get in doing it, 
the greater the likelihood that they will acquire the desired skills [Gronlund 1999; Mager 1997; Felder & 
Brent 1997]. An effective approach to achieving any desired learning outcome is therefore to show the 
students the course learning objectives that address that outcome, either on the first day of the course or 
(better) in study guides for the course tests. 

Other instructional techniques that address specific Criterion 3 outcomes are listed in Appendix 2. 
While a full discussion of all of the recommendations is beyond the scope of this paper, we have 
encountered enough faculty confusion in our workshops on EC 2000 regarding Outcome 3i (lifelong 
learning) to warrant explanation of our recommendations for this outcome. 

Candy [1991] defines lifelong learning skill as the ability to “continue one’s own self education 
beyond the end of formal schooling.” Drawing on work of McCombs [1991], Marra et al. [1999] suggest 
that if students are to be motivated and equipped to continue teaching themselves, their formal education 
must go beyond presentation of subject content to address four objectives: (1) helping them to understand 
their own learning processes, (2) requiring them to take responsibility for their own learning, (3) creating 
an atmosphere that promotes confidence in their ability to succeed, and (4) helping them see schooling 
and education as personally relevant to their interests and goals. 

The instructional methods suggested in Appendix 2 for addressing Outcome 3/ are consistent with 
these goals (as are the learning objectives suggested in Appendix 1). Clearly establishing the relevance of 
course material to students’ prior knowledge and interests effectively promotes learning, as a vast body of 
pedagogical research affirms [Bransford et al. 2000; McKeachie 1999], Acquainting students with their 
learning styles is a direct and effective way to help them understand their learning process [Felder 
1993(b)], Assignments that require independent literature and web searches promote a sense of 
individual responsibility for learning and also help develop the skill to find and organize information in 
the absence of texts and course notes. Finally, any student-centered instructional approach such as 
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cooperative learning or problem-based learning that moves the locus of responsibility for learning from 
the instructor to the student obviously prepares students to learn in environments where there are no 
instructors, lecture notes, textbooks, or any of the other trappings of formal schooling. 

B. Cooperative Learning 

Cooperative learning (CL) is instruction that involves students working in teams to accomplish a common 
goal, under conditions that include the following elements [Johnson, Johnson, and Smith, 1998]: 

1 . Positive interdependence . Team members are obliged to rely on one another to achieve the goal. 

2. Individual accountability. All students in a group are held accountable for doing their share of the 
work and for mastery of all of the material to be learned. 

3. Face-to-face promotive interaction. Although some of the group work may be parceled out and done 
individually, some must be done interactively, with group members providing one another with 
feedback, challenging one another's conclusions and reasoning, and perhaps most importantly, 
teaching and encouraging one another. 

4. Appropriate use of collaborative skills. Students are encouraged and helped to develop and practice 
skills in communication, leadership, decision-making, conflict management, and other important 
aspects of effective teamwork. 

5. Group processing. Team members set group goals, periodically assess what they are doing well as a 
team, and identify changes they will make to function more effectively in the future. 

A large and rapidly growing body of research confirms the effectiveness of cooperative learning 
in higher education [Johnson et al. 2000. Springer et al. 1998; Terenzini et al. 2001], Relative to students 
taught traditionally — i.e., with instructor-centered lectures, individual assignments, and competitive 
grading— cooperatively taught students tend to exhibit higher academic achievement, greater persistence 
through graduation, better high-level reasoning and critical thinking skills, deeper understanding of 
learned material, lower levels of anxiety and stress, more positive and supportive relationships with peers, 
more positive attitudes toward subject areas, and higher self-esteem. A number of references outline 
steps that can be taken to satisfy the five defining criteria of cooperative learning and achieving the 
learning benefits just enumerated [Felder & Brent 1994; Felder & Brent 2001, Johnson et al. 1998, Millis 
& Cottell 1998], In Appendix 3 we outline cooperative learning methods that address Outcomes 3a-3k in 
the context of an engineering course. 

C. Problem-Based Learning 

Another well-established instructional approach, problem-based learning (PBL), can easily be adapted to 
address all eleven outcomes of Criterion 3. In PBL, entire courses and individual topics within courses 
are introduced with complex open-ended focus problems whose solutions will require the knowledge and 
skills that constitute the desired course learning outcomes [Edens 2000; Maricopa 2001; McMaster 
University 2001; Woods et al. 2000], The students (generally working in groups) carry out the following 
steps: 

1 . Attempt to write a clear problem definition statement. 

2. Hypothesize ways to obtain a solution. 

3. Identify (a) what they know, (b) what they need to know (both information and methods), (c) 
what they need to do. These lists are regularly updated as the students proceed through the 
solution process. 
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4. Prioritize learning needs, set learning goals and objectives, and allocate resources and (if teams 
are used) responsibilities. 

5. Carry out the necessary research and analysis and generate possible solutions (first seeing if the 
problem can be solved with currently known information), examine their “fit,” choose the most 
appropriate one, and defend the choice. 

6. Reflect critically on the new knowledge, the problem solution, and the effectiveness of the 
solution process used. 

The instructor serves as a resource in all stages of this process, but does not provide formal instruction 
until the students (possibly with some guidance) have generated a need for it in the context of the 
problem. Any teaching method may be used to provide the instruction, ranging from lecturing to full- 
scale cooperative learning. Relative to students taught conventionally, students taught using PBL acquire 
greater mastery of problem-solving, interpersonal, and lifelong learning skills and are more likely to adopt 
a deep (as opposed to surface or rote) approach to learning [Woods et al. 1997; Woods et al. 2000], 

PBL was developed in the early 1970s in the McMaster University Medical School and has 
achieved widespread adoption in medical education. Its strongest proponent in engineering has been 
Donald Woods of the McMaster University Department of Chemical Engineering, who has developed 
resources for both instructors and students engaged in PBL [McMaster University 2001; Woods et al. 
1997], Other implementations in engineering have been undertaken at Lafayette College, Dartmouth 
College, and MIT [Hendley 1996], A problem-based program has been developed at Virginia 
Commonwealth University in which multidisciplinary teams of students run an engineering consulting 
firm, finding solutions to real industrial problems and in some cases generating millions of dollars in 
savings for their clients [Huvard et al. 2001], 

Once problem-based learning has been adopted in a course, very little additional work must be 
done to address all of EC 2000 Outcomes 3a-3k. Focus problems may be chosen to involve any 
experimental or analytical technique, tool, technical or interpersonal skill, or professional or 
contemporary societal issue that the instructor chooses to address. Appendix 4 outlines possible 
approaches to using PBL to address Outcomes 3a-3k, and Maskell [1999] discusses issues of assessment 
in a PBL environment. 

VI. SUMMARY 

Suppose a program has defined its educational objectives and program outcomes, and that each 
educational objective and each of EC 2000 Outcomes 3a— 3k maps onto one or more of the program 
outcomes. We propose the following approach to meeting the requirements of Criterion 3. 

1. [Program level] Select outcome indicators (assessment measures) for each program outcome and 
define performance targets for each indicator. (See Section II-B.) One approach is to break each 
outcome into elements (abilities specified or implied in the outcome that would require separate 
assessments), select attributes of each element (student activities that demonstrate mastery of the 
abilities), and then select appropriate assessment measures for each attribute. Besterfield-Sacre et 
al. [2000(a)] provide an excellent guide to this process. 

2. [Program level] Identify the program core — the required courses in the program curriculum that 
will collectively be identified as addressing the knowledge, skills, and attitudes enumerated in the 
program outcomes. 

3. [Course level] For every course in the core, define observable core learning objectives that are 
guaranteed to be in place regardless of who happens to teach the course (Section III-B and 
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Appendix 1), and define assessment methods for each core objective (Section IV and Besterfield- 
Sacre et al. [2000(a)]). Each core learning objective should map onto one or more program 
outcomes, and all program outcomes should be addressed by learning objectives in several core 
courses — the more, the better. (Some programs have found it helpful to formulate course 
outcomes for each required course that include some program outcomes or outcome elements, and 
then formulate the course learning objectives to address the course outcomes.) 

4. [Course level] Prepare a course assessment matrix with columns for program outcomes and rows 
for core course learning objectives (Table 3). Place a 1, 2, or 3 in the matrix to indicate that an 
objective addresses an outcome marginally, moderately, or substantively, respectively. The 
entries should reflect a consensus of all faculty members who are likely to teach the course before 
the next accreditation visit. 

5. [Program level] Prepare a program assessment matrix with columns for program outcomes and 
rows for outcome indicators and required courses. Place a 1, 2, or 3 in the matrix to indicate that 
an outcome indicator or required course addresses an outcome marginally, moderately, or 
substantively, respectively, basing the entries for each course on an examination of course 
materials and the course assessment matrix by a faculty review committee. 

6. [Course level] Teach each course in a manner that addresses all of the targeted program outcomes 
(Appendices 2-4). Carry out the assessment methods selected in Step 2, and place asterisks next 
to the 1 ’s, 2’s, and 3’s in the course assessment matrix when a core learning objective is judged to 
have been met. 

7. [Program level] Implement the program outcome assessment methods selected in Step 1 and 
evaluate the performance targets. Insert asterisks next to the l’s, 2’s, and 3’s for an outcome 
indicator to indicate that the corresponding performance target has been met. If the assessment 
for a particular outcome indicates shortcomings or room for improvement, initiate appropriate 
actions to improve instruction in the relevant courses. The program assessment matrix should 
indicate which courses might be modified, and the course assessment matrix for each of those 
courses should suggest areas that need strengthening. Possible instructional modifications may 
be found in Sections IIIC-IIIE. 

We make no claim that this procedure is the optimal way to prepare for an ABET visit: many 
engineering schools have developed and been successful with other approaches. We simply believe that it 
is rational and consistent with both the letter and spirit of EC 2000, and we propose that engineering 
programs consider adapting it to their own needs and resources. 

Regardless of the programmatic approach adopted, however, individual faculty members must 
take responsibility for assuring that the program outcomes are met and that program outcome assessment 
results are used for continuous program improvement. Fulfilling this responsibility entails defining core 
course learning objectives that clearly address program outcomes, selecting and implementing assessment 
methods that address all the objectives, and teaching the courses in a way that promotes strong positive 
results of the assessments. Our hope is that the suggestions and examples presented in the body and 
Appendices 1-4 of this paper will provide a useful resource to professors engaged in this process. 
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Table 1 

EC 2000 Criterion 3 Outcomes 

Programs must demonstrate that their graduates have: 

3a: an ability to apply knowledge of mathematics, science, and engineering 

3b: an ability to design and conduct experiments, as well as analyze and interpret data [Hicks 1982] 

3c: an ability to design a system, component, or process to meet desired needs [Adams, 1991; 

Bucciarella 1994; Papanek 1995; Petroski 1985; Solen & Harb 1998] 

3d: an ability to function on multidisciplinary teams [Felder & Brent 1994; Felder & Brent 1996; 

Johnson et al., 1998; Kaufman et al. 2000; McGourty & De Meuse 2000; National Institute for 
Science Education (a); Smith 1999] 

3e: an ability to identify, formulate, and solve engineering problems [Bucciarella 1994; Deek et al., 

1999; Edens 2000; Felder 1987; Fogler & Leblanc 1994; Smith 1999; Starfield et al. 1994; Woods 
et al. 2000] 

3f: an understanding of professional and ethical responsibility [Adams 1991; Harris et al. 1995; Haws 

2001; Huvard et al. 2001; Seebauer and Barry 2001; Whitbeck & Flowers 1998] 

3g: an ability to communicate effectively [Adamy 1987; Beer & McMurrey 1997; Brent & Felder 

1992; Hult 1996] 

3h: the broad education necessary to understand impact of engineering solutions in a global and societal 

context [Adams 1991; Bucciarella 1994; Florman 1996; Leifer 1997; Papanek 1995; Pfatteicher 
2001; Pool 1997] 

3i: a recognition of the need for and an ability to engage in lifelong learning [Branscomb 1997; Candy 

1991; Evers et al. 1998; Felder 1993(a); Felder 1993(b); Longworth & Davies 1996; Marra et al. 
1999; McCombs 1991] 

3j: a knowledge of contemporary issues [Florman 1996; Papanek 1995; Pool 1997] 

3k: an ability to use the techniques, skills, and modem engineering tools necessary for engineering 

practice [Branscomb 1997] 
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Table 2 

COURSE OUTCOME ASSESSMENT MATRIX: CHE 205' 



Core Learning objectives 


Outcome 1 


Outcome 2 


Outcome 3 


Outcome 4 


Outcome 5 


Outcome 6 


Outcome 7 


- in one or two sentences, explain in clear jargon-free language 
specific gravity, purge stream, vapor pressure, dewpoint,... 










2 






- perform PVT calculations using ideal gas and real gas equations 
of state 


2 














- perform bubble-point, dew-point, and vapor-liquid equilibrium 
calculations using Raoult’s law 


2 














- given a liquid mixture of two species, use tabulated physical 
properties to identify feasible separation processes 


3 




1 










- determine the absorption capacities of different solvents for a 
gaseous pollutant from tabulated Henry’s law constants, and 
explain what else must be known to choose the best solvent 


3 






1 








- calculate internal energy and enthalpy changes for specified 
species undergoing specified state changes 


2 














- given a process description, draw & label a flowchart, do degree 
of freedom analysis, outline material and energy balance solution 
procedure, and solve for required quantities 


3 














- define the four stages of team functioning and the responsibilities 
of a team coordinator, recorder, checker, and process monitor 












2 




- describe and implement effective teamwork practices and 
strategies for dealing with non-cooperative team members 










2 


3 





1 = objective addresses outcome slightly, 2 = moderately, 3 — substantively 



Outcome 1: 

Outcome 2: 

Outcome 3: 
Outcome 4: 

Outcome 5: 

Outcome 6: 
Outcome 7: 



Ability to apply mathematical, scientific, and engineering principles to the identification, 
formulation, and solution of engineering problems 

Ability to design and conduct experiments & to analyze and interpret data using modem 
engineering tools and techniques 

Ability to design engineering processes and products to meet desired needs 
Ability to analyze important social and environmental problems and identify and discuss 
ways that engineers might contribute to solutions, including technological, economic, and 
ethical considerations in the analysis 

Ability to communicate effectively in both writing and speaking in a variety of 
professional contexts 

Ability to function effectively in both single-discipline and multidisciplinary teams 
Recognition of need for and ability to engage in lifelong learning 



Table 3 

PROGRAM OUTCOME ASSESSMENT MATRIX 1 



Outcome indicators & courses 


Outcome 1 


Outcome 2 


Outcome 3 


Outcome 4 


Outcome 5 


Outcome 6 


Outcome 7 


Portfolio 


3 


3 


3 


3 


3 


3 


2 


FE Exam 


3 














GPA 


1 














GPA in CHE 


2 














Design course: Project report 


2 




3 




3 


1 


2 


Design course: Oral presentation 


2 




3 




3 


2 


2 


Exit interviews with seniors 


2 


2 


2 


2 


2 


2 


2 


Alumni interviews 


2 


2 


2 


2 


2 


2 


2 


ENGR 101 (Freshman engineering) 






1 


1 


1 


1 


2 


CS 110 




1 












ENGL 112 (Freshman composition) 










1 






ENGL 365 (Technical writing) 










3 






CHE 205 


3 








2 


2 




CHE 311 


3 




1 










CHE 312 


3 




1 










CHE 315 


3 














CHE 316 


3 














CHE 330 (Engineering laboratory) 


2 


3 






2 


3 


2 


CHE 410 (Engineering & society) 








3 


2 




2 


CHE 446 


3 




1 










CHE 425 


3 




2 










CHE 450 


3 




2 










CHE 451 (Capstone design course) 


3 




3 




3 


3 


2 



*1 = objective addresses outcome slightly, 2 - moderately, 3 - substantively 



Outcome 1: 

Outcome 2: 

Outcome 3: 
Outcome 4: 

Outcome 5: 

Outcome 6: 
Outcome 7: 



Ability to apply mathematical, scientific, and engineering principles to the identification, 
formulation, and solution of engineering problems 

Ability to design and conduct experiments & to analyze and interpret data using modem 
engineering tools and techniques 

Ability to design engineering processes and products to meet desired needs 

Ability to analyze important social and environmental problems and identify and discuss 
ways that engineers might contribute to solutions, including technological, economic, and 
ethical considerations in the analysis 

Ability to communicate effectively in both writing and speaking in a variety of 
professional contexts 

Ability to function effectively in both single-discipline and multidisciplinary teams 
Recognition of need for and ability to engage in lifelong learning 
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Figure 1. Elements of Course Design 
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APPENDIX 1 

ILLUSTRATIVE LEARNING OBJECTIVES FOR EC 2000 OUTCOMES 3a-3k 



Outcome 3a (apply knowledge of mathematics, science, and engineering) 

Outcome 3k (use modern engineering techniques, skills, and tools). 

The student will be able to (insert the usual engineering course objectives.) 

Outcome 3b (design and conduct experiments, analyze and interpret data) 

The student will be able to 

© design an experiment to (insert one or more goals or functions) and report the results (insert 
specifications regarding the required scope and structure of the report). Variants of this objective 
could be used in traditional lecture courses as well as laboratory courses. 

© conduct (or simulate) an experiment to (insert specifications about the goals of the experiment) 
and report the results (insert specifications regarding the scope and structure of the report). 

© develop a mathematical model or computer simulation to correlate or interpret experimental 
results (insert specifications regarding the experiment and the data). The results may be real data 
from a laboratory experiment or hypothetical data given to students in a lecture course. 

© list and discuss several possible reasons for deviations between predicted and measured results 
for an experiment, choose the most likely reason and justify the choice, and formulate a method 
to validate the explanation. 

Outcome 3c (design a system, component, or process). 

The student will be able to 

© design a system (or component or process) to (insert one or more goals or functions) and report 
the results (insert specifications regarding the required scope and structure of the report). 
Variants of this objective could be included in traditional lecture courses (including the freshman 
engineering course) as well as the capstone design course. 

• Use engineering laboratory data to design or scale up a system (or component or process). 

© build a prototype of a design and demonstrate that it meets performance specifications. 

© list and discuss several possible reasons for deviations between predicted and measured results 
for an experiment or design, choose the most likely reason and justify the choice, and formulate a 
method to validate the explanation. 

Outcome 3d (function on multi-disciplinary teams). The student will be able to 

• identify the stages of team development and give examples of things teams do that are 
characteristic of each stage. 

© summarize effective strategies for dealing with a variety of interpersonal and communication 
problems that commonly arise in teamwork. Choose the best of several given strategies and 
justify the choice. 

© function effectively on a team, with effectiveness being determined by instructor observation, peer 
ratings, and self-assessment. 

• explain aspects of a completed project related to the different disciplines. 



Outcome 3e (identify, formulate, and solve engineering problems). The student will be able to 

e troubleshoot a faulty process or product (insert specifications regarding the nature of the process 
or product) and identify the most likely sources of the faults. 

® create and solve homework or test problems and identify their levels on Bloom ’s Taxonomy. 

« examine a description of a problematic technology-related situation and identify ways that 
engineers might contribute to a solution. 

Outcome 3f (understand professional and ethical responsibility). Given a job-related scenario that 

requires a decision with ethical implications, the student will be able to 

• identify possible courses of action and discuss the pros and cons of each one. 

o decide on the best course of action and justify the decision. 

Outcome 3g (communicate effectively). The student will be able to 

• critique writing samples and identify both strong points and points that could be improved 
regarding grammar, clarity, and organization. 

e critique oral presentations and identify both strengths and areas for improvement. 

• write an effective memo (or letter, abstract, executive summary, project report) or give an 
effective oral presentation,... (insert specifications regarding the length and purpose of the 
communication and the intended audience). 

Outcome 3h (understand the global/societal impact of engineering solutions) 

The student will be able to 

• discuss historical situations in which technology had a major impact on society, either positively 
or negatively or both, and speculate on ways that negative results might have been avoided. 

Outcome 3i (recognize the need for life-long learning and be able to engage in it). The student will 

be able to 

• find relevant sources of information about a specified topic in the library and on the World Wide 
Web (or perform a full literature search). 

• identify his or her learning style and describe its strengths and weaknesses. Develop strategies 
for overcoming the weaknesses. 

• participate effectively in a team project and assess the strengths and weaknesses of the individual 
team members (including himself or herself) and the team as a unit. 

Outcome 3j (know contemporary issues). 

The student will be able to 

• identify an important contemporary regional, national, or global problem that involves 
engineering. Discuss ways engineers might make important contributions to the solution of the 

problem. 



APPENDIX 2 

TEACHING METHODS THAT ADDRESS OUTCOMES 3a-3k 



Outcome 3a (apply knowledge of mathematics, science, and engineering). All teaching methods 

customarily used in engineering education address this outcome. 

Outcome 3b (design and conduct experiments, analyze and interpret data) 

« Run a small number of experiments in engineering laboratory courses but make them open-ended, 
calling on the students to design and then carry out experiments to achieve specified goals rather 
than having them step through pre-packaged experiments. 

® Prescribe a lab report format that includes sections on experimental design, experimental 
procedures, instrument calibration and data analysis (including error estimation), and 
interpretation of results in light of theory. Have students critique good and bad examples of each 
section. 

• Provide study guides in laboratory courses with learning objectives that cover all aspects of the 
experiments (design, operation, data analysis and interpretation). Give individual tests consistent 
with the study guides. 

• Give students in lecture courses real or hypothetical experimental data to analyze and interpret. 
Build realistic experimental error into the data and sometimes give experimental results that 
contradict their expectations. Include problems of this nature in study guides and on tests. 

• Assign students in lecture courses to design experiments to measure specified variables and have 
them provide examples of the data they would expect to collect and how they would analyze it. 

Outcome 3c (design a system, component, or process) 

• Include design problems in courses throughout the curriculum (including the freshman 
engineering course). Early in the curriculum, provide constructive feedback and models of good 
responses to these problems but give them relatively little weight in grading, and increase their 
importance in grading as the students progress toward the senior year. 

® In all courses in which design problems are assigned (including the capstone design course), 
provide study guides with learning objectives that deal with every aspect of the process used to 
solve the problems. Give individual tests consistent with the study guides. 

• Bring experienced design engineers into engineering classes to talk about what they do and to 
give examples. 

• Use structured cooperative learning if designs are to be done by teams (details in next section). 

Outcome 3d (function on multidisciplinary teams) 

• In courses throughout the curriculum (starting with the freshman engineering course and 
including the capstone design course), assign projects that involve material and methods from 
different disciplines — e.g., different branches of engineering and physical sciences, biological 
sciences, mathematical science, computer science, economics, and management science. Form 
teams and assign team members to be responsible for the portions of the project associated with 
the different disciplines (if the students actually come from different disciplines, so much the 
better). 

• Provide training in effective team functioning. 
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• Provide study guides with learning objectives that cover elements of effective multidisciplinary 
team functioning (including strategies for cross-disciplinary communication and ways of dealing 
with common team dysfunctionalities), and give individual tests consistent with the guides. 

® Use structured cooperative learning, especially jigsaw (details in next section). 

Outcome 3e (identify, formulate, and solve engineering problems) 

• Include problem identification and formulation in course learning objectives. 

o In design or analysis problems in class, on assignments, and on tests, hypothesize situations in 
which the equipment or process in question is operated as stated in the problem but does not meet 
specifications and ask the students to brainstorm possible reasons for the discrepancy between 
predicted and measured performance. For example, after the students in a fluid dynamics course 

have determined that a -hp centrifugal pump should be adequate to deliver gal/min of a 

coolant from a storage tank through a system of pipe segments and fittings that span a specified 
vertical rise, tell them that the pump is installed and fails to achieve the specified delivery rate 
and ask for possible reasons. (Responses to such questions might include computational errors, 
measurement errors, instrument calibration errors, violations of assumptions or inappropriate 
approximations or failure to account for important factors in the design calculation, flaws in the 
purchased equipment, incorrect choice of model, algorithm, or formula, sabotage, etc.) 

• As part of a homework assignment, ask students to make up a problem having to do with the 
material taught in class that week. Tell them that they will get a minimal passing grade for a 
completely straightforward formula substitution problem and to get a higher grade their problem 
must call for deep understanding or critical or creative thinking on the part of the problem solver. 
Provide constructive feedback and examples of good responses. In a subsequent assignment, ask 
them to make up and solve a problem having to do with that week’s material, and later ask them 
to make up and solve a problem having to do with what they covered this week in this class and 
in some other class in the curriculum (multidisciplinary thinking), or one that involves an ethical 
dilemma (Outcome 3f) or a contemporary issue (Outcome 3h or 3j). Make copies of some or all 
student-generated problems for assessment purposes and consider including good ones on course 
tests. (Announce your intention of doing so when the assignment is given.) 

Outcome 3f (understand professional and ethical responsibility) 

• Include elements of ethical and professional responsibility in course learning objectives and on 
tests in at least one core engineering course in each year of the curriculum, including the capstone 
design course. Provide instruction in engineering ethics in the form of lectures or supplementary 
handouts. (A less effective alternative is to offer an elective course on professional and ethical 
responsibility.) 

• Include several course-related professional/ethical dilemmas in each engineering course that has 
professional and ethical issues in its learning objectives. Have students formulate responses and 
justifications individually, then reach consensus in pairs or teams of three. Provide constructive 
feedback and several alternative models of good responses. (Be sure to convey the idea that there 
is not one “correct” response, and that what matters is the clarity and logical consistency of the 
justification (Pfatteicher, 2001).) Have the students reformulate their initial responses to the 
dilemmas in light of the feedback. 

Outcome 3g (communicate effectively) 

• Incorporate “writing across the curriculum” or “writing to learn” methods into engineering 
courses. 
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e in learning objectives, in-class exercises and homework, and tests in lecture courses, include 
some qualitative descriptive problems (“Explain in terms a high school senior could understand 
the concept of .”) Grade both technical correctness and clarity of expression. 

• In courses that require technical report writing or oral presentation, provide preliminary 
instruction. Offer bad examples for students to critique and good and bad examples for them to 
compare and contrast. 

o Have students (or student teams) critique first drafts or presentations of other students’ (teams’) 
reports, including both technical content and clarity and quality of the written or oral 
communication. For written reports, collect but do not grade the first drafts; for written and oral 
reports, grade both the critiques and the revised draft or final presentation. 

Outcome 3h (understand impact of engineering solutions in a global/societal context) 

Outcome 3j (know contemporary issues). 

Put some in-class exercises, homework problems, and/or analyses of case studies that involve current 
global/societal issues into several engineering courses, including freshman engineering and capstone 
design courses. (Recent newspaper articles and science and society texts are good sources of topics.) 
Include such issues as environmental/economic tradeoffs, health and safety/economic tradeoffs, problems 
related to globalization such as movement of production facilities to other countries, total quality 
management, and pros and cons of government regulation of private industry. Ask students to generate 
potential solutions and evaluate them. Require such discussions as part of all major design projects. (Less 
effective — include a “Science and Society” course in the curriculum.) 

Outcome 3i (recognize need for and be able to engage in lifelong learning) 

• Clearly establish the relevance of new course material and assignments to phenomena and problems 
students already know about or might be called upon to deal with as professionals. 

• Teach students about learning styles, help them identify the strengths and weaknesses of their style & 
give them strategies to improve their study and learning skills [Felder 1993(b)], 

• Require library and Web searches and documentation of references. Grade on the thoroughness of 
the searches and the quality of the documentation. 

• Use active, cooperative, and problem-based learning, all approaches that move students away from 
relying on professors as the sole source of information and accustom them to relying on themselves 
and one another. 

• In general, anything done to meet Criteria 3e (identify and formulate engineering problems), 3 f 
(understand professional and ethical responsibility), and 3 /j (understanding of global/societal context 
of engineering solutions) automatically meets Criterion 3 i. 

Outcome 3k (Use modern engineering techniques, skills, and tools) 

• Have students use state-of-the-art technology for engineering system design, control, and analysis, 
mathematical analysis, web-based research, writing, and communication to the greatest extent 
appropriate. 

• Use computer simulations to conduct extensive parametric studies, process optimization, and “what- 
if ’ explorations. 

• Use modem equipment and instrumentation in undergraduate laboratories. 

• Include plant visits and presentations by practicing engineers in required engineering courses to make 
students aware of modem engineering tools and practices. 
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APPENDIX 3 

COOPERATIVE LEARNING METHODS THAT ADDRESS OUTCOMES 3a-3k 

To use cooperative learning, the instructor should have some or all course assignments (problem sets, 
laboratory experiments, design projects) done by teams of students that remain together for at least one 
month and as much as the entire semester. Roles should be defined for team members that rotate from one 
problem set, lab experiment, or phase of the project to the next. Possible roles are listed below: 

• (All settings) Coordinator (schedules meetings, makes sure all team members know what they are 
supposed to be doing and deadlines for doing it), recorder (coordinates preparation of the final 
solution set, lab report, or project report to be graded and of any required intermediate drafts), checker 
(verifies correctness of the final product), group process monitor (verifies that each team member 
understands each part of the final product, not just the part for which he or she was primarily 
responsible). 

• (Laboratory course) Experimental designer (coordinates determination of the data to be collected in 
each run, the number of runs to be carried out, the conditions of each run, and the required data 
analysis), operations supervisor and safety monitor (coordinates instrument calibration and operation 
and data recording), data analyst/statistician (coordinates data analysis, including estimation of error, 
and statistical quality control), data interpreter (coordinates interpretation of results in light of 
existing theory and/or material in related lecture courses). 

• (Design course) Process or product designer (coordinates conceptual design), process analyst 
(coordinates determination of process equipment and product specifications), production engineer 
(coordinates design of instrumentation, process control, and quality control systems and production 
planning and scheduling), economic analyst (coordinates cost and profitability analysis and process 
optimization). 

Two sets of roles may be assigned simultaneously, e.g., (a) and (b) in laboratory courses and (a) and (c) in 
design courses. 

The principal method of assuring individual accountability in cooperative learning is to give 
individual examinations covering every aspect of the assignment or project, something routinely done in 
lecture courses but rarely in laboratory or design courses. Another method applicable to courses 
involving oral project reports is to arbitrarily designate which team member presents which part of the 
report a short time before the reports are to be presented. The team members who were principally 
responsible for particular aspects of the project (for example, the occupants of the roles specified in Items 
(b) and (c) of the list given above) then have the added responsibility of making sure that all of their 
teammates understand what they did, and their project grade depends on their ability to provide that 
instruction. A third method is to collect peer ratings of team citizenship, construct weighting factors from 
the ratings, and apply them to team assignment grades to determine individual grades for each assignment 
[Kaufman et al. 2000). This procedure addresses many of the commonly expressed concern about team 
members who do not pull their weight on the team (and perhaps don’t participate at all) but receive the 
same grade as their more responsible teammates. Standard references on cooperative learning suggest 
other methods of achieving individual accountability and satisfying the other defining criteria for 
cooperative learning [Felder & Brent 1994; Felder & Brent 2001; Johnson et al. 1998, Millis & Cottell 
1998). 

When technical roles are assigned as in (b) and (c), thz jigsaw technique can be used to further 
enhance the learning benefits of cooperative learning. Once the teams have been formed and the roles 
assigned, “expert groups” consisting of all of the students in a specific role are given supplementary 
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training in their areas of expertise by a faculty member or graduate teaching assistant. In a laboratory 
course, for example, the operations supervisors (and no other team members) would be given instruction 
on operation of the experimental equipment, the data analysts would be given instruction on elements of 
error analysis and/or statistical quality control, and so on. Each team member has the responsibility of 
applying his or her expert knowledge to completion of the team assignment, thus assuring positive 
interdependence (if an expert does a poor job, everyone’s grade is diminished). 

Following are cooperative learning methods that specifically address Outcomes 3a-3k. 

Outcome 3a (apply knowledge of mathematics, science, and engineering). A large body of research 
data indicates that using cooperative learning in a course with mathematics, science, and engineering 
content increases the likelihood that this content will be mastered [Johnson et al. 2000; Springer et al 
1998; Terenzini et al 2001]. No specific technique is required to achieve this outcome as long as the five 
defining criteria of cooperative learning are met. 



Outcome 3b (design and conduct experiments, analyze and interpret data), Outcome 3c (design a 
system, component, or process), and Outcome 3d (function on multidisciplinary teams). Assign 
roles to laboratory team members that involve experimental design, analysis, and interpretation (3b), to 
design team members that involve all principal aspects of the design process (3c), and to team members in 
any project-based course that involve tasks commonly associated with different disciplines. Use jigsaw. 
Take measures to hold all team members individually accountable for every part of the final project 
report. 

Outcome 3e (identify, formulate, and solve engineering problems). At the beginning of a course, give 
a diagnostic assignment to assess skill in problem identification and formulation, and include on each 
homework team at least one individual who scored well on this assignment. Give team assignments that 
call for problem identification, formulation, and solution, followed by individual assignments and/or 
examinations that do the same. 

Outcome 3f (understand professional and ethical responsibility). Give assignments in which 
individuals analyze professional or ethical dilemmas (ideally imbedded within technical assignments) and 
then work in teams to reach consensus on how to respond to the dilemmas. Later in the course, include 
dilemmas in individual assignments and/or examinations. 

Outcome 3g (communicate effectively). Cooperative learning requires communication, and any of the 
techniques suggested in the cooperative learning literature to promote the success of the method 
automatically promotes the improvement of communication skills. When assignments involve written or 
oral communication, an effective technique is to have pairs of teams critique each other s first drafts of 
written reports or rehearsals of oral reports. The critiquing team members individually filling out copies 
of the rating sheet to be used for the actual evaluations and then reconcile their ratings and discuss them 
with the presenting team, which makes revisions taking the feedback into account. 

A communication technique for helping students resolve the serious disagreements and conflicts 
that sometimes arise in teamwork is active listening [ Felder & Brent 2001]. Have one side make its case, 
and then' have someone on the other side repeat the case verbatim without attempting to refute it, with 
people on the first side making corrections as needed until the party of the second part gets it right. Then 
the second side makes its case, and the first side has to repeat it without editorial comment. Finally, both 
sides try to work out an agreement that addresses everyone’s issues and feelings. 

Outcome 3h (understand impact of engineering solutions in a global/societal context). Use 
structured controversy [Johnson & Johnson 1995] to analyze case studies of controversial engineering 
solutions that have had a global or societal impact. Give each team member or pair of team members a 
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position or possible alternative solution to advocate. Give the advocates material to help them develop 
arguments for their position (or have them do their own research, which will also address Outcome 3i) 
and have them argue their positions in an intra-team debate. Then have them work as a team to formulate 
and justify a consensus position. 

Outcome 3i (recognize need for and be able to engage in lifelong learning). Using cooperative 
learning in any way at all moves students away from depending on teachers as resources and toward 
relying on themselves and their peers, the principal resources for lifelong learning. Having to work in CL 
teams promotes recognition of the need for independent and interdependent work; the experience of 
working in CL teams promotes the ability to do so successfully. 

Outcome 3j (know contemporary issues). Require teams to make up problems that place course content 
in the context of contemporary issues (which also addresses Outcome 3e). The issues may relate to 
professional or ethical dilemmas (Outcome 3f ) and/or global or societal issues (Outcome 3h). In 
subsequent assignments, have teams solve other teams’ problems. 

Outcome 3k (use modern engineering techniques, skills, and tools). In any group, some students are 
likely to have greater computing skills than their teammates have. If computer applications are included in 
course assignments done by cooperative learning teams, the novices will benefit from one-on-one tutoring 
from their more experienced colleagues and the latter students will receive the depth of learning that 
results from teaching others. The same argument can be made for any engineering technique, skill, or 
tool. 
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APPENDIX 4 

PROBLEM-BASED LEARNING METHODS THAT ADDRESS OUTCOMES 3a-3k 



Outcome 3a (apply knowledge of mathematics, science, and engineering). The traditional 
instructional approach in science, mathematics, engineering and technology that presents “fundamentals” 
and then (as much as three years later) presents the applications that make use of the fundamentals has 
repeatedly been associated with low motivation, poor learning, negative attitudes toward the subject, and 
high student attrition [Tobias 1990], Virtually all modem research-based references on effective teaching 
and learning agree that students have greater motivation to leam and leam more effectively when they 
perceive a need to know the material being taught [Felder et al. 2000; Ramsden 1994], Establishing a 
need to know material before teaching the material is almost by definition what problem-based learning 
does. 

Outcome 3b (design and conduct experiments, analyze and interpret data). Rather than having 
student teams work through a large number of pre-designed experiments in the engineering laboratory 
course, assign a small number of problems that require experimentation to solve (choosing problems that 
can be solved with either currently available or readily obtainable with existing resources), and have the 
student teams devise and implement experiments to solve them. Provide instruction or resources for self- 
study in experimental design, statistical data analysis, instrument calibration, equipment operation, etc., 
only after the teams have encountered a need to know the material. 

Outcome 3c (design a system, component, or process). In the capstone design course, do not provide 
instruction or resources for self-study in the elements of the design process — conceptual design, cost and 
profitability analysis, CAD, optimization, etc. — until the student teams encounter a need for instruction in 
those topics in the course of developing their designs. 

Outcome 3d (function on multidisciplinary teams). Assign problems whose solutions require material 
from several disciplines. (It would be difficult to find problems with the requisite complexity and open- 
endedness that fail to satisfy this condition.) Follow the recommendations for disciplinary role 
assignment given in the section on cooperative learning, making sure to hold all team members 
accountable for the work done by each of them. 

Outcome 3e (identify, formulate, and solve engineering problems). Problem-based learning is an 
ideal instructional approach for helping students develop skills in problem identification, formulation, and 
solution in that it explicitly requires students to do all three in the course of analyzing complex problems: 
simply using PBL is a major step in addressing this outcome. To further facilitate development of 
problem formulation skills, have students formulate their own focus problems once they have acquired 
some experience on instructor-formulated problems. 

Outcome 3f (understand professional and ethical responsibility). Incorporate professional and ethical 
dilemmas in focus problems. To impart a unique understanding of professional responsibilities, use a 
variant of the Virginia Commonwealth University student consulting team experience [Huvard et al. 
2001 ], 

Outcome 3g (communicate effectively). Development of communication skills occurs automatically in 
problem-based learning as long as written or oral reporting is part of the implementation procedure, 
especially if students work on the problems in structured teams. The greatest benefit is obtained if the 
implementation adheres to the principles of cooperative learning delineated in the preceding section. 

Outcome 3h (understand impact of engineering solutions in a global/societal context). Choosing 
PBL focus problems that have global or societal implications is perhaps the most effective way of 
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addressing this outcome. For example, assign the students to design a small, inexpensive, easily portable 
solar-powered water purification system for use in rural areas in developing countries and to explore its 
potential technical and economic benefits. 

Outcome 3i (recognize need for and be able to engage in lifelong learning). Any instructional method 
that transfers some of the burden of learning from the instructor to the students gives students an 
awareness of the need to assume this burden and helps them develop their skills at doing so. Problem- 
based learning is quintessentially a student-centered instructional approach, and the complex open-ended 
problems that provide the basis of the approach are exactly the types of problems the curriculum should 
be preparing the students to address throughout their careers. 

Outcome 3j (know contemporary issues). If focus problems involve contemporary issues, the students 
will end by knowing the issues to an extent that no other educational experience could provide. 

Outcome 3k (use modern engineering techniques, skills, and tools). As stated previously, focus 
problems can be chosen to address any technique, skill, or tool that the instructor wishes to address. 
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